Onsite Colombo Full Time
We are now hiring Lead Data Engineer – Data for the largest telecommunications partner in Sri Lanka. To be a part of a thriving team, please refer to the details below, and if you think you are qualified to be a good fit, send us your resume.
The Job
· Design, develop, and deploy efficient data pipelines on the existing cloud data engineering stack for collecting and organizing data.
· Monitor ETL/ELTs , carry out issue fixing and recoveries, ensuring SLAs
· Maintain and optimize the existing data model.
· Carryout data engineering development for data acquisitions, data model changes.
· Proactively identify pertaining issues, analyze causes, and recommend solutions.
· Collaborate with support engineers and the wider data community of the organization to provide transparency on the status of data uploads.
· Build required monitoring reports, dashboards, and alerts for the internal and data community's requirements.
· Carry out testing of new features for expanding the data engineering stack.
The Person
· 1 -3 years of hands-on experience in designing and developing big data solutions with excellent coding skills (Python, Java or Scala)
· Very good SQL skills
· Hands-on experience in scripting such as Shell Scripts and SQL.
· Fundamental understanding of data warehouses & data pipelines. Knowledge on SQL Database and NoSQL databases, such as Snowflake, CouchDB, MongoDB, Cassandra, HBase, DynamoDB.
· Experience with open source big data technologies such as Hadoop, Hive, Kafka and Spark on tuning, troubleshooting and scaling.
· Familiarity with one of more cloud-based big data platforms, such as Amazon EMR and Data Bricks
· Experience working with orchestration tools such as Apache Airflow, Azkaban, Apache Oozie, etc.
· Exposure in building real-time processing pipelines using Spark-streaming or Flink.
· Experience working with large data sets and analyzing huge volumes of data would be an added advantage.
· Prior experience working in an Agile environment would be an added advantage.